#!/bin/bash
Bash Shell Scripting Basics
Creating Bash Shell Scripts
Executing Bash Shell Scripts
Displaying Output
Using Variables
Using Bash Shell Expansion Features
Iterating With the for Loop
Troubleshooting Shell Script Bugs
Simple system administration tasks can be accomplished with Linux command-line tools
More complex tasks require chaining multiple commands
Command line tools can be combined with Bash shell to create powerful shell scripts
Bash shell script is executable file composed of list of commands
Can be further leveraged by other scripts
Proficiency in shell scripting is essential to Linux system administration
Especially crucial in enterprise environments
Shell scripts can improve efficiency and accuracy of routine tasks
Bash shell scripting not right tool for all scenarios
Programming languages have strengths and weaknesses; none is right for every situation
Bash scripts are good for tasks that can be accomplished mainly by calling other command line utilities
If task involves heavy data processing and manipulation, languages such as Perl or Python are better
Bash supports arithmetic operations limited to simple integer arithmetic
For more complex arithmetic operations, consider C or C++
Bash supports 1D and associative arrays, but Perl and Python have better array functionality
Gain experience with shell scripting and programming languages to learn advantages and disadvantages of each
Create Bash shell script by opening new file in any text editor
Advanced editors such as vim or emacs understand Bash shell syntax and provide color-coded highlighting
Highlighting is helpful for spotting syntax errors like unpaired quotes and unclosed brackets
First line of Bash shell script begins with 2-byte notation #!
Commonly referred to as a "sharp-bang" or "sha-bang"
Technically referred to as "magic pattern"
#! indicates that file is executable shell script
Path name follows to command interpreter that executes script
Bash scripts are interpreted by Bash shell and so begin with:
#!/bin/bash
Use chmod and possibly chown to change script file permissions and ownership so script is executable
Grant execute permission only to users script is intended for
Executable script can be invoked by name on command line
If only base name of script file is entered, Bash searches directories in shell PATH variable for first executable file matching name
Avoid script names that match other executable files
Configure PATH variable correctly so script is first match
To display directory script resides in, use which
[student@server1 ~]$ which hello ~/bin/hello [student@server1 ~]$ echo $PATH /usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/student/.local/bin:/home/student/bin
echoUse echo to display arbitrary text by passing text as argument to echo
By default, text is directed to STDOUT (standard out)
Can be redirected to STDERR (standard error)
Example: Use echo to display "Hello, world" to STDOUT
[student@server1 ~]$ cat hello #!/bin/bash echo "Hello, world" [student@server1 ~]$ ./hello Hello, world
Use echo in shell scripts to display information or error messages during script execution
Helpful indicator of script progress
Can be directed to STDOUT or STDERR or redirected to log file
Good practice to direct errors to STDERR to differentiate error messages from status messages
[student@server1 ~]$ cat hello #!/bin/bash echo "Hello, world" echo "ERROR: Houston, we have a problem." >&2 [student@server1 ~]$ ./hello 2> hello.log Hello, world [student@server1 ~]$ cat hello.log ERROR: Houston, we have a problem.
echo is useful for script debugging
Add echo statements to misbehaving part of script to clarify commands being executed and values of variables invoked
Some characters and words have special meanings to Bash shell
Some situations require literal values rather than the special meanings of these characters
Example: # is interpreted by Bash as beginning of comment and is therefore ignored along with everything following on same line
If "comment" meaning is not desired, then Bash needs to know that # is to be treated as a literal value
To disable meaning of special characters and words, use escape character, \, single quotes, '', or double quotes, ""
\ removes special meaning of single character immediately following
[student@server1 ~]$ echo # not a comment [student@server1 ~]$ echo \# not a comment # not a comment
To escape more than one character, use \ multiple times or use ''
Single quotes preserve literal meaning of all characters they enclose
[student@server1 ~]$ echo # not a comment # [student@server1 ~]$ echo \# not a comment # # not a comment [student@server1 ~]$ echo \# not a comment \# # not a comment # [student@server1 ~]$ echo '# not a comment #' # not a comment #
Double quotes are like single quotes but do not preserve literal value of dollar sign ($), back-ticks (`) and backslash (\)
The special meaning of \ is retained only when it precedes a dollar sign ($), back-tick (`), double quote ("), backslash (\), or newline
[student@server1 ~]$ echo '$HOME' $HOME [student@server1 ~]$ echo '`pwd`' `pwd` [student@server1 ~]$ echo '"Hello, world"' "Hello, world" [student@server1 ~]$ echo "$HOME" /home/student [student@server1 ~]$ echo "`pwd`" /home/student [student@server1 ~]$ echo ""Hello, world"" Hello, world [student@server1 ~]$ echo "\$HOME" $HOME [student@server1 ~]$ echo "\`pwd\`" `pwd` [student@server1 ~]$ echo "\"Hello, world\"" "Hello, world"
Useful in complex scripts
Serve as containers in which shell script stores data
Stored data easy to access and modify during script execution
To assign value to variable, use this syntax:
VARIABLENAME=value
Variable names are typically uppercase letters
Can also be numbers, lowercase letters, and underscore character, _
Cannot start with number
Use = without spaces to assign values
Example: Valid variables
COUNT=40 first_name=John file1=/tmp/abc _ID=RH123
Integer and string values commonly stored in variables
Good practice to quote string values when assigning to variables
Space character is interpreted by Bash as word separator when not enclosed in single or double quotes
Use of single or double quotes depends on how special characters should be treated
full_name='John Doe' full_name="$FIRST $LAST" price='$1'
To recall value of variable, precede variable name with $
Process known as variable expansion
Value of VARIABLENAME can be referenced with $VARIABLENAME
$VARIABLENAME syntax is simplified version of ${VARIABLENAME}
There are situations where brace-quoted form must be used
[student@server1 ~]$ FIRST_=Jane
[student@server1 ~]$ FIRST=John
[student@server1 ~]$ LAST=Doe
[student@server1 ~]$ echo $FIRST_$LAST
JaneDoe
[student@server1 ~]$ echo ${FIRST}_$LAST
John_DoeReplaces invocation of command with output of its execution
Allows output to be used in new context such as argument to another command, value of variable, and for loop construct (covered later)
Can be invoked by enclosing command in back-ticks: `<COMMAND>`
Preferred method is newer $() syntax: $(<COMMAND>)
[student@server1 ~]$ echo "Current time: `date`" Current time is Thu Jun 5 16:24:24 EDT 2014. [student@server1 ~]$ echo "Current time: $(date)" Current time is Thu Jun 5 16:24:30 EDT 2014.
Newer syntax preferred because it allows nesting command substitutions
Example: Use output of find as arguments for tar and store that output in variable TAROUTPUT
[root@server1 ~]# TAROUTPUT=$(tar cvf /tmp/incremental_backup.tar $(find /etc -type f -mtime -1)) [root@server1 ~]# echo $TAROUTPUT /etc/group /etc/gshadow /etc/shadow- /etc/passwd /etc/shadow /etc/passwd- /etc/tuned/active_profile /etc/rht /etc/group- /etc/gshadow- /etc/resolv.conf
Use to perform simple integer arithmetic operations
Syntax is $[<EXPRESSION>]
Arithmetic expressions enclosed within $[] are replaced with results
Before evaluation, Bash performs variable expansion and command substitution
Nesting of arithmetic substitutions is allowed
[student@server1 ~]$ echo $[1+1] 2 [student@server1 ~]$ echo $[2*2] 4 [student@server1 ~]$ COUNT=1; echo $[$[$COUNT+1]*2] 4
Space characters are allowed
Can improve readability in complicated expressions and when variables are used
[student@server1 ~]$ SEC_PER_MIN=60 [student@server1 ~]$ MIN_PER_HR=60 [student@server1 ~]$ HR_PER_DAY=24 [student@server1 ~]$ SEC_PER_DAY=$[ $SEC_PER_MIN * $MIN_PER_HR * $HR_PER_DAY ] [student@server1 ~]$ echo "There are $SEC_PER_DAY seconds in a day." There are 86400 seconds in a day.
Operator | Meaning |
<VARIABLE>++ | Variable post-increment |
<VARIABLE>-- | Variable post-decrement |
++<VARIABLE> | Variable pre-increment |
--<VARIABLE> | Variable pre-decrement |
- | Unary minus |
+ | Unary plus |
** | Exponentiation |
* | Multiplication |
/ | Division |
% | Remainder |
+ | Addition |
- | Subtraction |
Multiple operators evaluated according to precedence
To change evaluation order from default, use parentheses to group sub-expressions
[student@server1 ~]$ echo $[ 1 + 1 * 2] 3 [student@server1 ~]$ echo $[ (1 +1) * 2 ] 4
Order of precedence from highest to lowest, with equal operators listed together
Operator | Meaning |
<VARIABLE>++, <VARIABLE>-- | Variable post-increment and post-decrement |
++<VARIABLE>, --<VARIABLE> | Variable pre-increment and pre-decrement |
-, | Unary minus and plus |
** | Exponentiation |
*, /, % | Multiplication, division, remainder |
+, - | Addition, subtraction |
for LoopRepetitive task can take form of action executed multiple times on single target
Example: Checking every minute for 10 minutes to see if process has completed
Can also take form of action executed single time on multiple targets
Example: Backing up each database on system
for loop is shell looping construct used for task iterations
for <VARIABLE> in <LIST>; do
<COMMAND>
...
<COMMAND> referencing <VARIABLE>
doneLoop processes <LIST> items in order and exits
Each list item stored as value of <VARIABLE> while loop executes <COMMAND>
Naming of variable is arbitrary
<VARIABLE> value is typically referenced by <COMMAND>
for LoopCan enter for loop items directly
Can generate for loop items from shell expansions, such as variable substitution
Example: Different ways that lists can be provided to for loops
[student@server1 ~]$ for HOST in host1 host2 host3; do echo $HOST; done
host1
host2
host3
[student@server1 ~]$ for HOST in host{1,2,3}; do echo $HOST; done
host1
host2
host3
[student@server1 ~]$ for HOST in host{1..3}; do echo $HOST; done
host1
host2
host3
[student@server1 ~]$ for FILE in file*; do ls $FILE; done
filea
fileb
filec
[student@server1 ~]$ for FILE in file{a..c}; do ls $FILE; done
filea
fileb
filec
[student@server1 ~]$ for PACKAGE in $(rpm -qa | grep kernel); do echo "$PACKAGE was installed on $(date -d @$(rpm -q --qf "%{INSTALLTIME}\n" $PACKAGE))"; done
abrt-addon-kerneloops-2.1.11-12.el7.x86_64 was installed on Tue Apr 22 00:09:07 EDT 2014
kernel-3.10.0-121.el7.x86_64 was installed on Thu Apr 10 15:27:52 EDT 2014
kernel-tools-3.10.0-121.el7.x86_64 was installed on Thu Apr 10 15:28:01 EDT 2014
kernel-tools-libs-3.10.0-121.el7.x86_64 was installed on Thu Apr 10 15:26:22 EDT 2014
[student@server1 ~]$ for EVEN in $(seq 2 2 8); do echo "$EVEN"; done; echo "Who do we appreciate?"
2
4
6
8
Who do we appreciate?Typically due to typographical errors, syntactical errors, and poor logic
Easiest way to prevent is to catch them during authoring
Use text editor with syntactical highlighting
Develop good script-authoring practices and adhere to them
Use comments at beginning of script to explain script purpose, intended actions, and general logic
Use comments throughout script to clarify important or complex sections
Comments help when debugging and serve as reminders of script mechanics after time has passed
As long as syntax is correct, command interpreter executes commands without regard for structure and formatting
Break long commands into multiple lines of smaller code chunks
Easier to read and comprehend
Align beginning and end of multiline statements to show where control structures begin and end
Easier to see if statement is closed properly
Indent lines with multiline statements to represent hierarchy of code logic and flow of control structures
Use line spacing to separate command blocks to clarify beginning and end of code sections
Be consistent with formatting throughout script
#!/bin/bash
for PACKAGE in $(rpm -qa | grep kernel); do echo "$PACKAGE was installed on $(date -d @$(rpm -q --qf "%{INSTALLTIME}\n" $PACKAGE))"; done#!/bin/bash
#
# This script queries the RPM database to get information about when
# kernel-related packages were installed on a system.
#
# Variables
PACKAGETYPE=kernel
PACKAGES=$(rpm -qa | grep $PACKAGETYPE)
# Loop through packages
for PACKAGE in $PACKAGES; do
# Determine package install date and time
INSTALLEPOCH=$(rpm -q --qf "%{INSTALLTIME}\n" $PACKAGE)
# RPM reports time in epoch, so need to convert
# it to date and time format with date command
INSTALLDATETIME=$(date -d @$INSTALLEPOCH)
# Print message
echo "$PACKAGE was installed on $INSTALLDATETIME"
doneDo not make assumptions about integrity of inputs such as command-line arguments, user input, command substitutions, variable expansions, and file name expansions
Use proper quoting and sanity checking
Do not make assumptions about actions external to script such as interacting with files and calling external commands
Use Bash file and directory tests
Perform error checking on exit status of commands
Ruling out assumptions can keep script from failing
Example: Lines of code that make risky assumptions
cd $TMPDIR rm *
If directory change fails, file removal is performed on list of unknown files in unintended directory!
Not everyone agrees on what good practices are
It is important to apply guidelines consistently
Be aware of individual differences in programming styles and script formatting
When modifying another’s script, follow structure, formatting, and practices used by author
Imposing your style may make script inconsistent and reduce its readability and maintainability
To activate debug mode, add -x to command interpreter in first line of script
#!/bin/bash -x
Another way is to execute script as argument to Bash with -x
[student@server1 bin]$ bash -x <SCRIPTNAME>
Debug mode prints script commands and shell expansions before executing
[student@server1 bin]$ cat filesize
#!/bin/bash
DIR=/home/student/tmp
for FILE in $DIR/*; do
echo "File $FILE is $(stat --printf='%s' $FILE) bytes."
done
[student@server1 bin]$ ./filesize
File /home/student/tmp/filea is 133 bytes.
File /home/student/tmp/fileb is 266 bytes.
File /home/student/tmp/filec is 399 bytes.
[student@server1 bin]$ bash -x ./filesize
+ DIR=/home/student/tmp
+ for FILE in '$DIR/*'
++ stat --printf=%s /home/student/tmp/filea
+ echo 'File /home/student/tmp/filea is 133 bytes.'
File /home/student/tmp/filea is 133 bytes.
+ for FILE in '$DIR/*'
++ stat --printf=%s /home/student/tmp/fileb
+ echo 'File /home/student/tmp/fileb is 266 bytes.'
File /home/student/tmp/fileb is 266 bytes.
+ for FILE in '$DIR/*'
++ stat --printf=%s /home/student/tmp/filec
+ echo 'File /home/student/tmp/filec is 399 bytes.'
File /home/student/tmp/filec is 399 bytes.Large output of debug mode may hinder troubleshooting long scripts
Debug mode can be enabled for only part of script
Useful when source of problem is narrowed down to portion of script
To turned debugging on/off at specific points in script, use set -x and set +x
This example shows previous example script with debugging enabled only for command line enclosed in for loop
[student@server1 bin]$ cat filesize
#!/bin/bash
DIR=/home/student/tmp
for FILE in $DIR/*; do
set -x
echo "File $FILE is $(stat --printf='%s' $FILE) bytes."
set +x
done
[student@server1 bin]$ ./filesize
++ stat --printf=%s /home/student/tmp/filea
+ echo 'File /home/student/tmp/filea is 133 bytes.'
File /home/student/tmp/filea is 133 bytes.
+ set +x
++ stat --printf=%s /home/student/tmp/fileb
+ echo 'File /home/student/tmp/fileb is 266 bytes.'
File /home/student/tmp/fileb is 266 bytes.
+ set +x
++ stat --printf=%s /home/student/tmp/filec
+ echo 'File /home/student/tmp/filec is 399 bytes.'
File /home/student/tmp/filec is 399 bytes.
+ set +xTo invoke verbose mode, use -v
Bash prints each command to STDOUT prior to execution
To turn verbose mode on and off at specific points in script, use set -v and set +v
[student@server1 bin]$ cat filesize
#!/bin/bash
DIR=/home/student/tmp
for FILE in $DIR/*; do
echo "File $FILE is $(stat --printf='%s' $FILE) bytes."
done
[student@server1 bin]$ bash -v ./filesize
stat --printf='%s' $FILE) bytes."
stat --printf='%s' $FILE) bytes.
stat --printf='%s' $FILE
File /home/student/tmp/filea is 133 bytes.
stat --printf='%s' $FILE) bytes."
stat --printf='%s' $FILE) bytes.
stat --printf='%s' $FILE
File /home/student/tmp/fileb is 266 bytes.
stat --printf='%s' $FILE) bytes."
stat --printf='%s' $FILE) bytes.
stat --printf='%s' $FILE
File /home/student/tmp/filec is 399 bytes.Man pages: bash(1), magic(5), echo(1), echo(1p), and seq(1)
Bash Shell Scripting Basics
Creating Bash Shell Scripts
Executing Bash Shell Scripts
Displaying Output
Using Variables
Using Bash Shell Expansion Features
Iterating With the for Loop
Troubleshooting Shell Script Bugs
Nice job!
Click the button below to complete this module of the course:
Click the button below to continue to the course homepage:
Please continue with the next item in the course.